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ABSTRACT 

Factors predicting vocabulary learning and reading comprehension of advanced language learners of English in a 
linear multimedia text were investigated in the current study. Predictor variables of interest were multimedia 
type, reading proficiency, learning styles, topic interest and background knowledge about the topic. The outcome 
variables of interest were vocabulary and reading comprehension scores. Participants were 69 undergraduates 
enrolled at the foreign language teaching department of a Turkish university. Participants were randomly 
assigned to three different forms of an authentic electronic text, which differed from each other based on the type 
of multimedia: (a) definition of words, (b) definitions coupled with pictures, and (c) definitions coupled with 
short movies. The participants were given the text to read for general comprehension and were given an 
unannounced vocabulary test along with a reading comprehension test. Multiple regression analyses with 
vocabulary scores and reading scores as the criterion variables and the independent variables as the predictors 
served to reveal whether a relationship existed between the independent and dependent variables. Findings 
suggest that annotation type, reading ability and prior topical knowledge are important variables contributing to 
vocabulary learning whilst reading ability and learning styles (visual score) are important variables contributing 
to reading comprehension in a hypermedia environment. 

Keywords: Multimedia/Hypermedia, media in education, computer assisted language learning, vocabulary 
learning, reading comprehension 

OZET 

Bu cali^ma ileri diizey dil ogrencilerinin dogmsal hipermetin okuma ortamlannda sozciik ve okuma puanlarina 
etki eden faktorleri incelemektedir. Bagimsiz degi^kenler aciklayici not thru, okuma bilgisi, bili^sel ve algisal 
ogrenme stilleri, konuya ilgi ve on bilgi testlerinin sonuflaridir. Bagimli degi§kenler, sozciik ve okuma 
testlerinin sonuflaridir. Ara§tirmaya Tiirkiye’deki bir universitede okuyan 69 adet 1. simf Ingilizce ogretmenligi 
ogrencisi katilmi§tir. Katihmcilar, ozgiin bir elektronik metni okumak amaciyla yansiz atama yoluyla he gruba 
ayrilmisjlardir. Elektronik metinler birbirlerinden a£iklayici notlarm thru bakimindan farkhdir. U? farkli 
a£iklayici not kullamlmr^tir: a) Sozciik tammlari, b) sozciik tammlan ve bu tammlarla ilgili resimler, c) sozciik 
tammlan ve bu tammlarla ilgili kisa filmier. Katilimcilardan anlama amaciyla metni okumalan isten mi§tir. Daha 
sonra onceden duyurulmami§ bir sozciik testi ile beraber okuma testi verilmi§tir. Degi§kenler arasindaki ili§kiyi 
gormek amaciyla yapilan coklu regresyon analizine sozciik ve okuma testlerinin sonuflan bagimli degi§ken 
olarak, yukanda sayilan diger degi§kenler ise bagimsiz degi^kenler olarak dahil edilmi§tir. Sonuclar, aciklayici 
not thru, okuma bilgisi ve konuyla ilgili on bilginin hipermetinde sozciik ogrenmeye katkisi olan, ote yandan 
okuma bilgisi ve gorsel ogrenme stillerinin okuma-anlamaya katkisi olan degi§kenler oldugunu gostermi§tir. 

Anahtar Kelimeler: ('ok lu ortamlar, egitimde medya, bilgisayar destekli dil egitimi, sozciik ogrenme, okuma- 
anlama 

INTRODUCTION 

Using multimedia or hypermedia as a supplemental medium for foreign/second language (L2) learning and 
teaching has received considerable interest. Notable numbers of multimedia software have been developed for 
teaching foreign languages since multimedia is considered a promising technology to facilitate L2 learning and 
teaching (Chiquito, Meskill, & Renjilian-Burgy, 1997). Due to its integration of different media, multimedia 
could have positive effects on language learning (Kramsch & Andersen, 1999; Shea, 1996). Along with the 
presentation of input in multiple forms, interaction provided by hypermedia and the use of authentic materials 
can make it an ideal language learning medium (Chiquito et ah, 1997; Heller, 1990; Kramsch & Andersen, 
1999). 

Multimodality in hypermedia environments is basically provided with the aid of hypermedia annotations or 
glosses incorporated into a reading text in multiple forms of media. It is suggested that providing glossaries is 
better than simplifying a text in helping learners to cope with comprehension since simplification deteriorates the 
authenticity of the text (Luppescu & Day, 1993; Watanabe, 1997; Widdowson, 1984). Moreover, glosses 
increase flow of reading, independence from dictionaries and accuracy in finding meaning (Nation, 1990). 
Finally, they help readers confirm or disconfirm their vocabulary guesses from the context. 
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Research studies on the effectiveness of hypermedia on L2 learning have mostly investigated the effects of 
multimedia glossing techniques on certain aspects of L2 learning such as vocabulary learning (Chun & Plass, 
1996a; De Ridder, 2003; Hulstijn, Hollander, & Greidanus, 1996; Knight; 1994; Seghayer, 2001) and text 
comprehension (Ariew & Ergetin, 2004; Aust, Kelley, & Roby, 1993; Chun & Plass, 1995, 1996b; Davis & 
Lyman-Hager, 1997; Ercetin; 2003; Knight, 1994; Lomicka, 1998; Roby, 1999; Sakar & Ercetin, 2005). 

With respect to vocabulary learning, previous studies suggest that glosses/annotations with word definitions and 
visuals (e.g., pictures and videos) help learners learn vocabulary more efficiently than annotations with word 
definitions alone (Chun & Plass, 1996a; Seghayer, 2001). However, empirical studies have revealed 
contradictory findings regarding the relative importance of the type of visual annotations. Chun and Plass 
(1996a) suggest that annotations which include word definitions coupled with pictures are more effective on 
vocabulary learning than annotations that include definitions coupled with videos. Seghayer’s (2001) 
investigation, on the other hand, has yielded the conclusion that definitions coupled with videos are more 
effective in facilitating vocabulary learning. 

When certain variables are added into the research design, notable changes in the results can occur. For instance, 
Knight (1994) investigated the effects of on-line dictionary access along with L2 verbal ability on vocabulary 
learning of learners of French. Results indicated that vocabulary learning outcomes varied according to the 
linguistic proficiency of the learners. 

As for reading comprehension, it is suggested that hypermedia environments have the potential to facilitate L2 
reading comprehension since they provide additional information at both the word and the topical level 
(Lomicka, 1998; Martinez-Lage, 1997). In other words, electronic reading provides L2 readers with the 
opportunity to learn beyond the texts via textual and extratextual information contrary to the traditional reading. 
However, the relationship between hypermedia use and reading comprehension is somewhat indirect (Chun, 
2001). Other variables, such as verbal ability and proficiency may interact with the effects of hypermedia (Ariew 
& Ercetin. 2004). These predictions stem from the reading studies conducted in LI and L2. These studies 
revealed that the ability to utilize better reading strategies to comprehend the text depends on the learners' 
foreign language reading proficiency (Devine, 1988), prior knowledge on the subject domain (Carrell, 1988), 
topic interest (Schiefele & Krapp, 1996) and learning styles (Bernhardt, 1991). 

The variables that affect traditional reading and vocabulary learning are expected to be influential in hypermedia 
environments as well. Furthermore, providing several authentic multimedia symbol systems should create a 
more interesting learning environment serving learners with different learning styles. It is suggested that 
differences in learning styles should result in distinctive navigation patterns and differences in learning outcomes 
in hypermedia environments (Parkinson & Redmond. 2002). Literature provides studies with contradictory 
findings some establishing a relationship between individual learning styles and learning outcomes (Andris, 
1996; Plass, Chun, Mayer, & Leutner, 1998) whilst others failing to establish a connection between learning 
styles and achievement (Calcaterra, Antonietti, & Underwood, 2004; Liu & Reed, 1994). Replication of the 
studies in different learning situations and with different learner populations may help to shed light on these 
issues. The current design aims to investigate whether selected explanatory variables predict foreign language 
vocabulary learning and reading comprehension in hypermedia environments at a practically significant level. 

METHOD 

Participants 

The experiment was conducted with 69 (22 males and 47 females) freshman TEFL students studying at an 
English-medium university in Turkey. In order to encourage participation in the experiment, full participation 
was awarded with 5 % of students’ English Composition grades. Participants were randomly assigned to three 
groups each containing 23 students. 

Materials 

Treatment: Reading Software 

Using the reading software designed by Ariew (1999), a hypermedia text was generated. The software allowed 
integration of glosses to an electronic text collecting data with its built-in tracking tool, which recorded every 
single interaction of the readers with the material. That is, the frequencies of access to annotations, the order in 
which they are accessed and the amount of time spend on reading were recorded by the tracking tool. 

The original text of the software comes from an advanced reading book by Smith and Mare (1997). The reading 
passage was chosen based on a topic interest questionnaire given to participants before. A pen-and-paper version 
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of the original text was piloted with 20 senior students who did not participate in the original study in order to 
determine the words to be annotated. All unknown words were annotated. For each word, 4 pictures and 3 videos 
were found. Four English teachers selected the most suitable videos and pictures for each word. Each picture was 
standardized to a size of 450 pixels width and 303 pixels height. The videos used were similar to each other in 
length and quality. The mean length of the videos was 13.7 seconds (SD: 1.07). The American Heritage 
Dictionary (2000) was used as the source of all the word definitions. 

Three parallel forms of the electtonic text were prepared. The first form included only textual definitions, which 
provided the definition of a highlighted word or phrase, and its grammatical form -noun, adjective, verb, etc 
appears. For example, the textual annotation for the word “doomed” appears as ‘adj. - Condemned to ruination 
or death' . The second version of the text included an associated image of the word in addition to the definition 
provided in the first group. The third version of the text included an associated movie and the definition provided 
in other groups. In picture and video groups, participants were free to choose either one type of annotation or 
both types provided. Annotations were presented successively when they were clicked rather than 
simultaneously. 

The text was presented in a linear order in 9 pages and students were allowed to go forward and backward in the 
text by clicking on one of two arrow-buttons provided at the bottom of the screen. The design had a navigation 
map at the top-left side of the screen that indicated which page the student was reading and how many pages 
there were left to read. 

Vocabulary Test 

Overall vocabulary scores were computed based on 3 subtests. They were prepared by the researcher based on 
the tests used by Knight (1994), Rott (1999) and Waring and Takaki (2003). All subtests were piloted for item 
development. Items with insufficient item facility, item discrimination and distractor efficiency indices were 
modified. The first subtest included a checklist in which students were asked to indicate the words they 
remembered from the text. The checklist was comprised of 42 target items and 30 fillers. The second subtest 
involved students writing the LI equivalents or synonyms of every target word. Since this subtest would allow 
subjectivity in scoring, two independent raters scored the papers and an interrater reliability of .95 was found for 
the pre-test, .98 for the immediate post-test and .98 for the delayed post-test. The third subtest was a multiple- 
choice test whose first half involved synonym recognition and the second half involved definition recognition. 
The reliability coefficients obtained from the actual administration revealed that alpha of the pre-test was .76, 
that of immediate post test was .72 and that of the delayed post test was .69. 

Reading Comprehension Test 

The reading comprehension test consisted of a multiple-choice and a true-false section. The true -false part 
consisted of 22 questions and each question had a third choice called “no information”. This reduced the chance 
of getting the right answer by guessing alone from 50 % to 33 %. The multiple -choice part consisted of 12 
questions, 4 distractors for each. In both parts, grammatically parallel and correct distractors were provided. 
Moreover, incidental insignificant information and details were not tested. The test was developed by the 
researcher with the assistance of an expert, and edited by an American colleague to eliminate odd structures or 
unidiomatic language. Administration procedures including speed and time control were determined after a pilot 
study. The actual data revealed that the Cronbach's Alpha of the multiple choice test was .79 and that of true- 
false part was .70. 

Prior Knowledge Test 

The prior knowledge test consisted of two parts. In the first part, participants were asked to write down every 
single thing they knew about the topic. A total of 31 propositions were mentioned by the participants in the 
essays and each coherent and acceptable proposition was given 1 point. An independent rater was trained and an 
interrater reliability coefficient of .97 was found between the scorers on the first part. The second part consisted 
of 5 open-ended questions and 2 precise completion items. The highest possible score was determined to be 16 
for the second part. A detailed scoring rubric was prepared and an independent rater was trained. An interrater 
reliability coefficient of .96 was found between the researcher and the independent scorer on the second part. 
The cumulative scores of the two parts were used as the prior knowledge score. 

Nelson - Denny Reading Test 

In order to see whether the groups were equal with regard to their reading ability, the Nelson-Denny Reading 
Test prepared by Brown, Fisho and Hanna (1993) was used. It is a standardized reading survey for high school 
and college students and adults, which measures vocabulary development, comprehension and reading rate. The 
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test was administered to all the participants at the same time and in the same room according to the instructions 
given in the Nelson-Denny test manual. 

Learning Style Questionnaire 

Rebecca Oxford's (1993) ‘Style Analysis Survey (SAS)’ was used as the learning style assessment instrument in 
the study. The material provides individuals’ overall style preferences focusing on their general approach to 
learning and working. SAS has a Cronbach Alpha of .87 in testing with 468 language students at the university 
level in a U.S. university. 

Topic Interest Questionnaire 

On the very same sheet with the prior knowledge test, a topic interest test was also given to the students. The test 
was adapted from Schiefele and Krapp (1996). The questionnaire consisted of value-related and feeling related 
valences. To calculate value -related valences, students were asked to rate the topic on its usefulness, 
meaningfulness, importance and worthiness on a 4-point scale, 4 showing complete agreement and 1 showing 
complete disagreement. To calculate feeling-related valences, students were asked how they would feel, when 
they read the text by rating again on a 4-point scale, 4 implying complete agreement with a specific feeling and 
1 implying complete disagreement with it; i.e., boredom, interest, indifference, involvement and engagement. 
For each subject, a score of topic interest was computed by adding feeling-related and value related scales 
(Schiefele & Krapp, 1996; 146). The highest possible score was 36. 

Procedure 

Participants were given the Nelson-Denny Reading test, the prior knowledge test and vocabulary pre-test two 
weeks before the treatment. They were given the learning style questionnaires a week before the treatment. The 
treatment was given at one of the computer laboratories of the university, which was designed for simultaneous 
processing of 25 computers each connected to a local area network (LAN). Participants were given the treatment 
successively on the very same day. Before the reading session, the researcher oriented students with the reading 
software by providing a demonstration on how the software worked through a data projector. Each group was 
given exactly the same instructions and was told that they should read the text carefully since they would have a 
reading comprehension test after the treatment session. After the treatment, participants were given the 
unannounced vocabulary test, the announced reading comprehension test and a background questionnaire 
successively. Within four days following the treatment 63 of the 69 students were given semi-structured 
interviews. Exactly 3 weeks after the treatment, again unannounced, all students were given the vocabulary test 
again. 

Data Analysis 

Since randomization was realized, the groups were considered to be theoretically equal (Ary, Jacobs, & 
Razavieh, 1996). However, a further inspection of the groups was also carried out. The groups were compared 
using one-way analyses of variance (ANOVA) on vocabulary pre-test scores, general point average (GPA), age, 
prior topical knowledge, topic interest, L2 reading ability, PC aptitude, WWW use, learning style, and frequency 
of lookup behavior to determine whether the groups differed from each other in terms of those variables at a 
probability value of .05 or below. The assumptions of ANOVA were checked before running the analyses. The 
groups were not different from each other at the inception of the study at a probability value of .05. 

In order to control for the vocabulary pre-test scores, gained vocabulary scores were calculated. All variables 
were put into a bivariate correlation calculation in SPSS 14.0 for windows. Explanatory variables that had 
moderate or high correlations with response variables were selected, and multiple regression analyses with each 
response variable were conducted. Assumptions of normally distributed errors, linearity, multicollinearity and 
uncorrelated residual term were checked before interpreting the analyses. 

RESULTS 

Vocabulary Learning 
Immediate Post-test Scores 

Predictor variables that had high correlation with gained vocabulary post-test scores were language proficiency 
(1-.323, p<.007) and prior topical knowledge (r=.355, p<.003). Text format was used as a dummy variable in the 
analysis. Variables were entered into the analysis successively. Language proficiency scores explained 10 % of 
the vocabulary post-test scores (R 2 =. 104, F=7.801, p<.007). The next variable, annotation type, created an R 
squared change of .16. More specifically, it explained an additional 16 % in the criterion variable (R 2 
change=.16, F change=7.076, p<.002). The final variable, prior knowledge score, explained an additional 4.6 % 
in the criterion variable (R 2 change=.046, F change=4.309, p<.042). The regression model with these three 
variables accounted for 31 % of variance in the criterion variable. In other words, above variables explained 31 




The Turkish Online Journal of Educational Technology - TOJET January 2007 ISSN: 1303-6521 volume 6 Issue 1 Article 5 



% of immediate vocabulary post-test scores. The regression equation is as follows: Immediate post-test score= 
0.258 (language proficiency) + 0.241 (annotation type) + 0.228 (prior topical knowledge). 

Further analysis revealed that the groups performed differently based on the annotation type (F 2> 66=9.819; 
p<.001). Multiple comparisons with Scheffe test revealed that both the picture and video group performed 
significantly better than the definition only group in terms of immediate post-test scores. However, there was no 
difference between the picture and the video group. 

Delayed Post-test Scores 

The only predictor variable that had high correlation with delayed vocabulary post-test scores was language 
proficiency (r=.3 19, p<.008). Even though prior topical knowledge had a significant correlation with the 
criterion variable (r=.278 p<.021), it did not create a significant R 2 value in the hierarchical regression analysis. 
Text format was used as a dummy variable in the analysis again. The analysis revealed that language proficiency 
scores explained 10 % of variability in the criterion variable (R~=.102, F=7.590, p<.008). Annotation type, 
created an R squared change of .12. In other words, it explained an additional 12 % in the criterion variable (R 2 
change=.116, F change=4.828, p<.011). Overall, the model with these two variables accounted for 22 % of 
variance in the criterion variable. More specifically, the variables explained 22 % of delayed vocabulary post- 
test. The following regression equation evolves: Delayed post-test score= 0.306 (language proficiency) + 0.163 
(annotation type). 

Further analysis was conducted with delayed post-test scores as well. Similar to above results, groups performed 
differently based on the annotation type (F 2 , 66 =4.822; p<.011). Multiple comparisons with Scheffe test revealed 
that the video group performed significantly better than the definition only group. 



Reading Comprehension 

Potential explanatory variables that had high correlation coefficients with the criterion variable were reading 
ability (1-.386, p<.001) and learning styles (SAS visual score) (r=.248, p<.04). Annotation type was used as a 
dummy variable. A hierarchical regression analysis with the reading comprehension scores as the criterion and 
above variables as the predictors was conducted. Results revealed that language proficiency scores explained 15 
% of variability in the criterion variable (R 2 =.149, F=1 1.760, p<.001). The second variable, SAS visual score, 
explained an additional 6 % variability in the criterion (R 2 change=.059, F change=4.887, p<.03). Annotation 
type did not have a significant R 2 value in the analysis. In sum, language proficiency and SAS visual score 
explained 21 % of variability in reading comprehension scores. The regression equation evolving from this 
analysis is as follows: Reading comprehension score= 0.383 (language proficiency) + 0.242 (SAS visual score). 

DISCUSSION 

Findings of the study suggest that when advanced language learners are under scrutiny, predictors of vocabulary 
learning in hypermedia reading environments are annotation type, reading ability and prior topical knowledge. 
Moreover, findings suggest that predictors of reading comprehension are reading ability and SAS visual score. 

These findings confirm the results of previous studies. More specifically annotation type is an important 
variable contributing to vocabulary learning (Chun & Plass, 1996a; Seghayer, 2001) along with linguistic 
proficiency (Knight, 1994). Moreover, as suggested by Devine (1988), the ability to comprehend the text 
primarily depends on the learners' foreign language reading proficiency. Finally, in terms of learning styles, the 
findings follow a path similar to those of Andris (1996) and Plass et al (1998) maintaining that there is a 
relationship between individual learning styles and learning outcomes. Within the scope of the current study, 
one could claim that hypermedia environments may provide learning opportunities for learners with different 
visual learning styles. 

Combining text with visuals regardless of the type of visual used is more effective in facilitating vocabulary 
learning than providing only definitions of words. In this respect, findings are in line with the Generative 
Theory of Multimedia Learning (Mayer, 1997; 2001), particularly with the dual channels assumption, which 
suggests that it is better to present an explanation in words and visuals than solely in words. 

Treatment of the study had low element interactivity, that is, elements in the material could be learnt 
successively rather than simultaneously since they did not interact. So, the intrinsic cognitive load was low. 
Sweller and his colleagues suggest that extraneous cognitive load is a problem in cases of high element 
interactivity. They claim that manipulating the instructional design might have no beneficial outcomes when the 
element interactivity is low (Sweller, 1994; Sweller, van Merrienboer, & Paas, 1998). The current study 
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suggested that students exposed to visual annotations were better in vocabulary tasks than those who were 
merely exposed to verbal annotations even though the element interactivity of the material is low. Thus, 
extraneous cognitive load could be a problem also for low element interactivity materials for advanced language 
learners. 

Several pedagogical implications can be suggested based on the current design and relevant literature. In 
hypermedia reading environments, the interaction between the reader and the text provides individualized 
learning and promotes learner autonomy. Learners can have control over their learning process and learn at their 
own pace. Therefore, both slow learners and fast learners can make use of hypermedia environments efficiently 
because of the interaction they provide. Second, the presentation of authentic input is made easier with 
hypermedia software. Besides, learners are not only presented with real language, but also with original means 
to deal with the authentic language. Thus, multimedia are more feasible in creating natural L2 learning tasks 
than text simplification methods that may result in inauthenticity. Third, in order to make better use of 
hypermedia environments, L2 readers need to add new strategies to their repertoire to be independent during the 
reading process. In this respect, familiarizing L2 learners with hypermedia experience and training them can be 
quite feasible. Finally, professionals involved in material development should consider the importance of visual 
aids in language teaching through hypermedia. 

The study poses several limitations; thus, the findings should be considered with caution. First, the target 
population of the study was EFL students who learn English for academic purposes. This study should be 
replicated in other learning contexts with students from different levels to generalize findings to a larger target 
population. Second, further studies with larger sample sizes must be conducted to investigate whether a really 
significant variation existed in the population. Third, the use of a pre-test affected the generalizability of this 
study: the results cannot be generalized to people who are not pre-tested. Finally, participants were exposed to a 
non-traditional treatment, but tested with traditional testing methods in the current study. Employing on-screen 
tests where visual elements are incorporated would be more suitable for use in hypermedia environments. 
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